FORM PTO-1390 
(REV 11 -2000) 



U S. DEPARTMENT OF COMMERCE PATENT AND TRADEMARK OFFICE 



,|f f 17Rtt'dPCr/FT0 U JUL 2001 



ATTORNEY'S DOCKET NUMBER 

124-872 

U S APPLICATION NO (If known, see 37 C F R 15) 



TRANSMITTAL LETTER TO THE UNITED STATES 
DESIGNATED/ELECTED OFFICE (DO/EO/US) 
CONCERNING A FILING UNDER 35 U.S.C. 371 



89040 



INTERNATIONAL APPLICATION NO. 
PCT/GBOO/00063^ 



INTERNATIONAL FILING DATE 

12 January 2000 / 



PRIORITY DATE CLAIMED 

13 January 1999 



TITLE OF INVENTION 



IMPROVED METHOD OF PROCESSING DATA AND DATA PROCESSING APPARATUS / 



APPLICANT(S) FOR DO/EO/US 



BARRETTTet aQ € ^ ^ C ^ rWl 



Applicant herewith submits to the United States Designated/Elected Office (DO/EO/US) the following items and other information 

1 . [X] This is a FIRST submission of items concerning a filing under 35 U.S.C. 371 . 

2. □ This is a SECOND or SUBSEQUENT submission of items concerning a filing under 35 U.S.C. 371 . 

3. [ 



This is an express request to begin national examination procedures (35 U.S.C. 371(f)). The submission must include 
items (5), (6), (9) and (21) indicated below. 



4. M The U.S. has been elected by the expiration of 19 months from the priority date (Article 31). 

5. t% copy of the International Application as filed (35 U.S.C. 371(c)(2)). 

^1. □ is attached hereto (required only if not communicated by the International Bureau). 
%. has been communicated by the International Bureau. 

X. □ is not required, as the application was filed in the United States Receiving Office (RO/US). 

6. C|] An English language translation of the International Application as filed (35 U.S.C. 371 (c)(2)). 
2|. □ is attached hereto. 

Tb. □ has been previously submitted under 35 U.S.C. 154(d)(4). 

7. CD Amendments to the claims of the International Application under PCT Article 1 9 (35 U.S.C. 371 (c)(3)) 
St. □ are attached hereto (required only if not communicated by the International Bureau). 

%%. □ have been communicated by the International Bureau. 

1% □ have not been made; however, the time limit for making such amendments has NOT expired. 

Ha. □ have not been made and will not be made. ' 

8. □ An English language translation of the amendments to the claims under PCT Article 19 (35 U.S.C. 371(c)(3)). 

9. □ An oath or declaration of the inventor(s) (35 U.S.C. 371(c)(4)). 

1 0. □ A English language translation of the annexes of the International Preliminary Examination Report under PCT 

Article 36 (35 U.S.C. 371(c)(5)). 



Items 11 To 20 below concern document(s) or information included: 



11. 


□ 


12. 


□ 


13. 




14. 


□ 


15. 


□ 


16. 


□ 


17. 


□ 


18. 


□ 


19. 


□ 


20. 





A substitute specification. 



540765 



ttmecmmo i t jh 2001 



U.S. APPLICATION WO (If known, see 37 CiR 1 .5) 




INTERNATIONAL APPLICATION NO. 

PCT/G BOO/00063 



ATTORNEY'S DOCKET NUMBER 

124-872 



21 . The following fees are submitted 



CALCULATIONS PTO USE ONLY 



BASIC NATIONAL FEE (37 C.F.R. 1 .492(a)(1 )-(5): 

-- Neither international preliminary examination fee (37 C.F.R. 1 .482) 
nor international search fee (37 C.F.R. 1 .445(a)(2)) paid to USPTO 
and International Search Report not prepared by the EPO or JPO $1 000.00 

-- International preliminary examination fee (37 C.F.R. 1 .482) not paid to 

USPTO but International Search Report prepared by the EPO or JPO $860.00 

- International preliminary examination fee (37 C.F.R. 1 .482) not paid to USPTO 

but international search fee (37 C.F.R. 1.445(a)(2)) paid to USPTO $710.00 

-- International preliminary examination fee (37 C.F.R. 1 .482) paid to USPTO 

but all claims did not satisfy provisions of PCT Article 33(1 )-(4) $690.00 

-- International preliminary examination fee (37 C.F.R. 1 .482) paid to USPTO 

and ail claims satisfied provisions of PCT Article 33(1 )-(4) $1 00.00 

ENTER APPROPRIATE BASIC FEE AMOUNT = 



Surcharge of $130.00 for furnishing the oath or declaration later than □ 20 
months from the earliest claimed priority date (37 C.F.R. 1 .492(e)). 



30 



860.00 



130.00 



CLAIMS 



NUMBER FILED 



NUMBER EXTRA 



RATE 



Totfei Claims 



32 



-20 = 



12 



$18.00 



216.00 



Independent Claims 



-3: 



$80.00 



160.00 



MlijriPLE DEPENDENT CLAIMS(S) (if applicable) 



$270.00 



0.00 



TOTAL OF ABOVE CALCULATIONS = 



1366.00 



□g Applicant claims small entity status. See 37 CFR 1 .27. The fees indicated above 
1= are reduced by 1/2. 



0.00 



SUBTOTAL = 



1366.00 



Processing fee of $1 30.00, for furnishing the English Translation later than □ 20 □ 30 
mdnths from the earliest claimed priority date (37 C.F.R. 1 .492(f)). + 



0.00 



TOTAL NATIONAL FEE = 



1366.00 



FgfSfor recording the enclosed assignment (37 C.F.R. 1 .21 (h)). The assignment must be 
aqgijmpanied by an appropriate-cover sheet (37 C.F.R. 3.28, 3.31). $40.00 per property 



0.00 



Fe^jfor Petition to Revive Unintentionally Abandoned Application ($1240.00 - Small Entity = $620.00) 



0.00 



TOTAL FEES ENCLOSED = 



1366.00 



Amount to be: 
refunded 



Charged 



a. 
b. 



d. 



to cover the above fees. 



EE! A check in the amount of $1 366.00 to cover the above fees is enclosed. 
O Please charge my Deposit Account No. 14-1 140 in the amount of $ 

A duplicate copy of this form is enclosed. 
I3 The Commissioner is hereby authorized to charge any additional fees which may be required, or credit any 

overpayment to Deposit Account No. 14-1140 . A duplicate copy of this form is enclosed. 
£3 The entire content of the foreign application(s), referred to in this application is/are hereby incorporated by reference in this 



application. 

NOTE: Where an appropriate time limit under 37 C.F.R. 1 .494 or 1 .495 has not been met 
or (b)) must be filed and granted to restore the application to pending status 



SEND ALL CORRESPONDENCE TO: 

NIXON & VANDERHYE P.C. 
1 100 North Glebe Road, 8 th Floor 
Arlington, Virginia 22201-4714 
Telephone: (703) 816-4000 



etition to revive (37 C.F.R. 1.137(a) 




Stanley C. Spooner 



NAME 



27,393 



July 11, 2001 



REGISTRATION NUMBER 



Date 



- 2- 



540765 



09/889040 

JC18 Rec'd PDT/PTO 1 t JUL 2001 

IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re Patent Application of 

BARRETT et al. Atty. Ref.: 124-872 

Serial No. Unknown Group: 

Filed: July 11,2001 Examiner: 

For: IMPROVED METHOD OF PROCESSING DATA AND DATA PROCESSFNG 
APPARATUS 

*T* *I* *T* *T* -7* rf* sfr JJ> ^ 3f* ifZ 

July 11,2001 

Assistant Commissioner for Patents 
Washington, DC 20231 

Sir: 

PRELIMINARY AMENDMENT 

In order to place the above-identified application in better condition for examination, 
please amend the application as follows: 

IN THE CLAIMS 

Please substitute the following amended claims for corresponding claims previously 
presented. A copy of the amended claims showing current revisions is attached. 

3. (Amended) A method according to claim 1 in which the equations comprise linear equations. 

4. (Amended) A method according to claim 1 in which the variables correspond to points in an 
image of a surface constructed from an array of data samples from different positions on the 
surface. 

5. (Amended) A method according to claim 1 in which each variable allocated to a bounded area 
corresponds to the parameter to be sensed (e.g. surface temperature). 
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6. (Amended) A method according to claim 1 in which the variables comprise surface 
temperature measurements and the sensor comprises a microwave antenna adapted to detect 
radiation emitted from or reflected off the surface. 

7. (Amended) A method according to claim 1 in which more than one set of linear equations is 
used and each linear equation in a set may be allocated a weighting, with different weightings 
used for each set. 

8. (Amended) A method according to claim 1 in which the sensor is adapted to receive 
information radiated or reflected from a footprint on the surface to produce a data sample and the 
linear equations may then be calculated from data samples corresponding to a number of such 
footprints for different surface areas. 

11. (Amended) A method according to claim 1 in which each bounded area is smaller than the 
size of each footprint for the data samples. 

12. (Amended) A method according to claim 1 in which each data sample used to construct the 
set of linear equations corresponds to a footprint covering a different but overlapping area of the 
surface. 

13. (Amended) A method according to claim 1 in which the boundaries are defined by 
reprojecting the data samples onto a GIS map of the surface. 

14. (Amended) A method according to claim 1 in which the weights, a, are based on the 
convolution for the gain function of the receiver with the boundaries of the area surface sensed. 
16. (Amended) A software program for implementing on a computer adapted to perform a 
method in accordance with claim 1. 
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19. (Amended) A method according to claim 17 in which the weighting coefficients are 
associated with the directional sensitivity of the sensor(s) used to capture the set of original 
signals. 

20. (Amended) A method according to claim 17 in which the processed signal for each mapping 
region is determined by assuming it to be a constant value and solving a redundant, or solvable, 
number of simultaneous equations which each have the processed signal as one parameter, and 
weighted values derived from the original signals as other parameters, the weighting and the 
original signals being known. 

21. (Amended) A method according to claim 17 in which the mapping regions are representative 
of real physical features known to be present in the scene being viewed by the image-gathering 
apparatus. 

24. (Amended) A method according to claim 22 in which the method comprises processing the 
original signals to obtain a plurality of values for at least some pixels of the original pixellated 
field of view. 

26. (Amended) A software carrier carrying software which when operational on a computer or 
network operates the computer or network according to the method of claim 1. 

29. (Amended) Apparatus according to claim 27 in which the image capturing means comprises 
a microwave antenna or an infrared receiver. 

30. (Amended) Apparatus according to claim 27 in which the image capturing means is mounted 
on a remote sensing vehicle such as a satellite, aeroplane or ship. 

31. (Amended) A satellite system, or other remote sensing platform or installation having one or 
more sensors having an overlapping field of view so as to produce overlapping unprocessed 
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images or data samples, and means for processing the images or data samples using a method in 
accordance with claim 1. 

REMARKS 

The above amendments are made to place the claims in a more traditional format. 
Attached hereto is a marked-up version of the changes made to the claims by the current 
amendment. The attached pages are captioned " Version With Markings To Show Changes 
Made.' 



Respectfully submitted, 




SCS:ms 

1 100 North Glebe Road, 8th Floor 
Arlington, VA 22201-4714 
Telephone: (703) 816-4000 
Facsimile: (703) 816-4100 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 
IN THE CLAIMS 

3. (Amended) A method according to claim 1 [or claim 2] in which the equations comprise linear 
equations. 

4. (Amended) A method according to [any preceding claim] claim 1 in which the variables 
correspond to points in an image of a surface constructed from an array of data samples from 
different positions on the surface. 

5. (Amended) A method according to [any preceding claim ] claim 1 in which each variable 
allocated to a bounded area corresponds to the parameter to be sensed (e.g. surface temperature). 

6. (Amended) A method according to [any preceding claim] claim 1 in which the variables 
comprise surface temperature measurements and the sensor comprises a microwave antenna 
adapted to detect radiation emitted from or reflected off the surface. 

7. (Amended) A method according to [any preceding claim] claim 1 in which more than one set 
of linear equations is used and each linear equation in a set may be allocated a weighting, with 
different weightings used for each set. 

8. (Amended) A method according to [any preceding claim] claim 1 in which the sensor is 
adapted to receive information radiated or reflected from a footprint on the surface to produce a 
data sample and the linear equations may then be calculated from data samples corresponding to 
a number of such footprints for different surface areas. 

11. (Amended) A method according to [any preceding claim] claim 1 in which each bounded 
area is smaller than the size of each footprint for the data samples. 
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12. (Amended) A method according to [any preceding claim] claim 1 in which each data sample 
used to construct the set of linear equations corresponds to a footprint covering a different but 
overlapping area of the surface. 

13. (Amended) A method according to [any preceding claim] claim 1 in which the boundaries 
are defined by reprojecting the data samples onto a GIS map of the surface. 

14. (Amended) A method according to [any preceding claim] claim 1 in which the weights, a, 
are based on the convolution for the gain function of the receiver with the boundaries of the area 
surface sensed. 

16. (Amended) A software program for implementing on a computer adapted to perform a 
method in accordance with [any one of claims 1 to 15] claim 1 . 

19. (Amended) A method according to claim 17 [or claim 18] in which the weighting 
coefficients are associated with the directional sensitivity of the sensor(s) used to capture the set 
of original signals. 

20. (Amended) A method according to [any one of claims 17 to 19] claim 17 in which the 
processed signal for each mapping region is determined by assuming it to be a constant value and 
solving a redundant, or solvable, number of simultaneous equations which each have the 
processed signal as one parameter, and weighted values derived from the original signals as other 
parameters, the weighting and the original signals being known. 

21. (Amended) A method according to [any one of claims 17 to 20] claim 17 in which the 
mapping regions are representative of real physical features known to be present in the scene 
being viewed by the image-gathering apparatus. 
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24. (Amended) A method according to claim 22 [or 23] in which the method comprises 
processing the original signals to obtain a plurality of values for at least some pixels of the 
original pixellated field of view. 

26. (Amended) A software carrier carrying software which when operational on a computer or 
network [either provides an image processing apparatus according to claim 25 or] operates the 
computer or network according to the method of [any one of claims] claim 1 [to 24]. 

29. (Amended) Apparatus according to claim 27 [or claim 28] in which the image capturing 
means comprises a microwave antenna or an infrared receiver. 

30. (Amended) Apparatus according to claim 27 [, 28 or 29] in which the image capturing means 
is mounted on a remote sensing vehicle such as a satellite, aeroplane or ship. 

31. (Amended) A satellite system, or other remote sensing platform or installation having one or 
more sensors having an overlapping field of view so as to produce overlapping unprocessed 
images or data samples, and means for processing the images or data samples using a method in 
accordance with [any one of claims 1 to 20] claim 1 . 
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1 

IMPROVED METHOD OF PRO CESSING DATA AND 
DATA PROCE SSING APPARATUS 

T 1 

This invention relates to improvements in methods of processing data, and 
5 especially (but not exclusively) to a method of extracting information 
from remotely sensed data. The invention also relates to data/image 
processing apparatus. 



Remote sensing is a field of activity with many applications. In a typical 

10 application, a sensor such as a microwave antenna on a satellite may be 
adapted to produce data from microwave signals emitted from the earth's 
surface or intermediate surfaces such as clouds or vegetation. Each data 
sample measured corresponds to the signal emitted from an area or 
footprint of the surface determined by the field of view of the sensor. As 

15 the satellite moves, data from different footprints are collected and can be 
processed to produce a data sample indicative of a spectral characteristic, 
such as brightness temperature, for a footprint, sets of data samples being 
used to generate a two-dimensional image of the surface. In one example, 
the image may correspond to the temperature of different parts of the 

20 sensed surface, such as land or sea which can be used for weather 
forecasting. Other applications include the measurement of wind speed at 
the ocean surface; precipitation; land and sea surface temperatures; ice 
and snow cover etc, etc. Obviously, the shape of the footprint will 
depend upon the satellite and sensor characteristics, shape of the surface 

25 and the location of each footprint on that surface. Similar considerations 
apply in the case of remote sensing by sensors mounted on other types of 
platforms, e.g. aircrafts, ships, and the ground. 

Typically, the sensed data sample corresponding to a footprint is 
30 individually processed and allocated to a data pixel. The pixel is selected 
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to be a location on the surface corresponding to the centre of the 
footprint. However, this approach is inefficient in its use of the 
information making up the data sample. 

5 In the case of apparent earth surface temperature measurements at 
microwave frequencies, the footprint for each sensed data sample may be 
of more than 1000km 2 (say 30km x 30 km). At coastal regions, where a 
footprint crosses both land and sea, contamination of detected signals 
occurs resulting in erroneous or inaccurate data samples. This is due to 

10 geophysical differences between land and sea. Prior systems have relied 
upon masking out the region where erroneous values occur, but this 
results in a loss of information around the coastline. Hitherto this has 
either been accepted as an inevitable result of preventing the 
contamination from giving unreliable results in that area, or linear 

15 extrapolation between reliable data either side of the sensed coastal area 
has been proposed. This is due to geophysical differences between land 
and sea. Similar considerations apply to any discontinuities across the 
entire range of scales. In order to overcome these problems, current 
solutions are to design better sensors, or orbit the satellite lower, or both. 

20 

In accordance with a first aspect, the invention provides a method of 
extracting information from remotely sensed data comprising the steps of: 

acquiring a plurality of data samples from at least one sensor, each 
25 data sample corresponding to a respective footprint and said one or 

more sensors being adapted to sense a parameter of a surface and 
having a known gain function; 
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processing (reprojecting) the data samples to produce a respective 
footprint corresponding to the area of the surface contributing to 
each data sample; 

selecting a set of n bounded areas for the surface being sensed and 
allocating a variable to the area defined by each boundary; 

defining a weighting a for each data sample dependent upon the 
sensor gain function and the location of the footprint for the 
respective data sample relative to the bounded areas of the surface; 

constructing a set of at least n equations from said weighted data 
samples; 

and solving said equations to determine the values for the variables 
allocated to respective bounded areas. 

Preferably each data sample comprises a convolution of the sensor gain 
function with the signal from the footprint on the surface which is 
incident upon the sensor. The data samples may form a data stream which 
may be defined spatially, or temporally or both. 

Preferably, the equations comprise linear equations. This makes them 
easier to solve, perhaps using matrixes. 

Looked at one way, each data sample in the stream may represent the net 
radiation measured by the sensor over its field of view, taking into 
account its varying sensitivity for different angles of incidence (its gain 
function). The area of surface contributing to the field of view is defined 
as the sensor footprint. 
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The variables may therefore correspond to attributes of the surface in an 
image (for example a substantially two-dimensional image) of a surface 
constructed from an array of data samples from different positions on the 
5 surface. 

Preferably, (n + m) or more linear equations are formulated where (n + 
m) is the number of data samples used and m is greater than or equal to 
zero (and integer). This will (generally) produce an over-specified 
10 system. 

Preferably, each variable allocated to a bounded area corresponds to the 
parameter to be sensed (e.g. surface temperature) and assumes that the 
variable is relatively constant (at least over the measurement period) 
15 within the boundary (or will be closer to the values for its bounded area 
than an adjacent bounded area, or other bounded areas). 

For example, the information extracted may comprise surface temperature 
measurements. In this case, the sensor may comprise a microwave or 
20 infrared sensor or antenna adapted to detect radiation emitted from or 
reflected off the surface. The variable will then be a temperature value. 
By solving the linear equations for all or part of the entire area set, the 
values can be calculated from the weighted data. 

25 By giving a weighting to each data sample, and using several data samples 
to generate a set of linear equations, it is possible to utilise more of the 
data to improve accuracy. In the past, in practice it has generally been 
assumed that the individual data samples will correspond to accurate 
measurements of the variable with respect to the sampling frequency. 

30 
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More than one set of linear equations may be used. Each linear equation 
in a set may be allocated a weighting, with different weightings used for 
each set. 

5 Preferably the sensor is adapted to receive information radiated or 
reflected from a footprint on the surface to produce a data sample rather 
than a single point measurement. The linear equations may then be 
calculated from data samples corresponding to a number of such footprints 
for different surface areas. A boresight may be provided at the sensor to 
10 limit the field of view of the sensor, optimising the footprint size. 

Preferably, the footprints for two or more of the data samples used to 
construct the set of linear equations overlap at least partially. The 
overlap may be (by area) 5% or 10%, or 50% or any other value which 
15 enables the equations to be solved. 

Two bounded areas may be chosen for the surface to be sensed. For 
example, these may define the boundaries of areas of land and ocean at a 
coastline. Of course, many more boundaries may be selected. In one 

20 envisaged application, the boundaries divide the surface into a grid, with 
each area in the grid being allocated a variable. Each area may be a 
square or other shape which can be identified. The area of each "square" 
of the grid may be smaller than the size of each footprint for the data 
samples. This enables an increased resolution to be obtained. For 

25 increasingly small grid areas, the amount of overlap between footprints 
may need to be increased to produce a computable set of equations. 

Preferably, each data sample used to construct the set of linear equations 
corresponds to a footprint covering a different but overlapping area of the 
30 surface. The sensor may therefore be moved relative to the surface and 
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take a series of snapshots of different areas over time. Alternatively, a 
set of snapshots may be taken during a single timespan. 

In one possible arrangement, nine data samples are used to construct nine 
5 linear equations, with the footprints for each data sample being arranged 
in a 3 row by 3 column grid. However, the optimal number of equations 
depends on the relationship between the sensor and the areas defined. 
Each footprint in the grid may partially overlap its adjacent footprint. 

10 In an alternate arrangement, enough data samples may be acquired to 
completely cover the surface with overlapping footprints. These data 
samples can then be used to provide a set of equations whereby the 
variables allocated to the bounded areas in the surface are all determined 
in one process. This is an alternate to using groups or sets of data 

15 samples to cover the surface in several sets. 

The sensor in one preferred arrangement may be adapted to produce a 
data sample for a footprint in the form: 

+°° 

where t is the time relative to the measurement time, h(t) is the impulse 
response of the sensor, G(Q,§) is the normalised gain function for the 
sensor in the direction (9,<j>) relative to the direction of the sensor at the 
25 measurement time, ®(t),<&(t) describe the movement of the sensor during 
the measurement period and (X(8,<j)),F(8,<J>))is the projection of the 
viewing direction (9,(J>) onto the surface. 
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In the case of microwave sensing, T 0 may be surface temperature 
brightness, although it could be any other parameter depending on the 
type of sensor used. It will also be appreciated that other parameters than 
land and sea temperature can be measured, such as snow field cover, 
ocean surface wind speed etc. 

Assuming that brightness temperatures vary slowly compared to the time 
scale of the measurement and reversing the order of integration: 

T a = f\G(Q,$)T b (X(e4),Y(94))cos$dQd<b 
where G(9,<j>) = jh(t)G(0(t) +6,4»(f) +$)dt 

G(0,<j>) is the sensor gain function integrated over the measurement 
interval. 

The footprint associated with the true antenna response G(9,<j>) is known 
as the Instantaneous Field of View (IFOV). The footprint for the time 
integrated G(Q,§) is called the Effective Field of View (EFOV). The 
processing of the data samples to project the EFOV onto the surface of the 
Earth, for example, results in an elliptical footprint, with the shorter axis 
parallel to the scan. If the sampling distance at each frequency is shorter 
than the dimensions of these projected footprints they overlap. 

Each equation may be (for two variables) of the form: 

T 0 (x,y) = T a L(x,y) + T b (l-L(x,y)) 

where L(x,y) = 1 if (x,y) is a point within a first boundary 
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= 0 if (x,y) is a point within a second boundary 
T a = variable allocated to the first boundary 

T b = variable allocated to the second boundary and x,y 

are co-ordinates representing a position on the 

surface. 

From nine data samples, T c (x,y), nine linear equations in terms of T a and 
T b can be established and solved for T a and T b . 

5 Convolving the antenna gain function over the model yields: 

T 0 =fjG(Q,q>){L(X(e4),Y(B,$>))T a + (l-L(X(Q4),Y(&4)))T t }co8$dBd$ 

=aT a + iX-a)T b 
where a = \\G(Q^)L{X{Q,^),Y{Q,^))cos^dM^ 

10 

Finally, restructuring enables the system of equals to be expressed as: 
T Dij =a uj T a + = 1,2,3/ = 1,2,3 

15 The nine equations can be expressed in a compact form as: 

T 0i =a u {T a -T b ) + T b 

where ct ; j are the weights for each data sample. 

20 

By overlaying the footprint for the central data sample of the grid of 3 x 3 
samples onto the map, the values of T a and T b derived (or other variables 
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as appropriate) can be allocated to each boundary area within that 
footprint. 

After evaluating the parameter for the central footprint, the calculation 
5 can be repeated by forming values of T a and T b from a second set of 
footprints offset from the first set. This can be continued to build up a 
two-dimensional image of the parameters across the whole or a substantial 
part of the surface. 

10 Obviously, the method can be readily expanded to cope with more than 
two parameters, such as land, sea and ice for instance, e.g. T a , T b , T c 
The boundaries may be defined by reprojecting the data samples onto a 
map of the surface, such as a GIS data model. The reprojection may be 
adapted to take into account changes in the profile of the surface, which 

15 may otherwise distort the footprint for each data samples. 

The weights, a, may be based on the convolution for the gain function of 
the receiver with the boundaries of the area surface sensed. The 
convolution may be approximated by a summation. 

20 

The linear equations may be solved using linear regression. They may be 
solved using matrix mathematical techniques. 

The data samples may, for example, comprise brightness temperature 
25 values, but may correspond to other parameters. They may be produced 
by integrating the sensor output over time, or be an instantaneous value. 
By selecting overlapping data samples to form a set, and using a grid of 
smaller areas than the sensor resolution (footprint size), increases in 
resolution can be obtained. A grid area for footprint area of V 8 or 
30 smaller could be used. 
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In accordance with a second aspect, the invention provides a software 
program for implementing on a computer adapted to perform a method in 
accordance with the first aspect of the invention. Alternatively, the 
5 method may be embedded in a computer chip to perform the data 
extraction in hardware. It may be adapted to process the data in real 
time. 

According to a third aspect, the invention provides a method of processing 
10 an original set of signals to create a processed image comprising: 

taking a set of original signals derived from one or more image sensors, 
the signals being representative of an image to be image processed; 

15 having a spatial map upon which the original image signals are to be 
translated, the map having a plurality of mapping regions; 

having weighting coefficients for each original signal in the set of original 
signals corresponding to the weight to be given to each individual original 
20 signal associated with any particular mapping region, and 

creating processed image signals for each mapping region, the processed 
image signals for individual mapping regions being determined by 
applying the weighting coefficients to the set of original signals for each 
25 individual mapping region so as to create a combined, weighted, 
processed signal associated with each mapping region. 

Preferably, there are at as least many original signals in a set as there are 
spatial mapping regions, and most preferably more original signals in a 
30 set than mapping regions. 
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Preferably, the weighting coefficients are associated with the directional 
sensitivity of the sensor(s) used to capture the set of original signals. 

5 Preferably, the processed signal for each mapping region is determined by 
assuming it to be a constant value and solving a redundant, or solvable, 
number of simultaneous equations which each have the processed signal as 
one parameter, and weighted values derived from the original signals as 
other parameters, the weighting and the original signals being known. 

10 

Preferably, some or all mapping regions represent known, predetermined 
shapes. By knowing in advance what the "shapes" and/or sizes of the 
mappings should be it is possible to solve the equations for the initially 
unknown processed signal value for each mapping region from the 
15 measured original signals and the weighting coefficients to be applied - 
possibly using a best-fit technique if there is no exact solution. 

The mapping regions may be standard, possibly equal, areas (for example 
a grid). The mapping regions may be irregular, with not all of them 

20 having the same areas, and/or outline shapes. Some or all of the mapping 
regions may be of precisely known areas/shapes representative of real 
physical features known to be present in the scene being viewed by the 
image-gathering apparatus. For example the mapping regions could be 
parts of a known coastline, or known fields, or individual geographic 

25 features. 

Looked at in another way the invention can be thought of as a new 
technique for creating high resolution imagery from low resolution sensor 
sampling in both space and time. 
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"Traditional" remote sensing consists of obtaining a set of sensor values. 
These are readings taken from a field of view seen, for example, by an 
observing satellite, and displaying these values as a grid of pixels on a 
computer display. The location of the centre of the field of view is 
5 computed from the satellite orbit and viewing geometry. This is then 
used to locate the appropriate display pixel. If more than one value is 
allocated to a pixel then some rule is used to select the "best" value. If 
the resulting image shows empty pixels then this is because not every field 
of view centre falls within every pixel, in this case some interpolation of 
10 adjacent data is required to supply missing pixel values. In order to 
minimise these effects the pixel image is scaled so that the pixel size 
approximates to size of the sensor field of view. In this way an image of 
the entire view scanned by the sensor is built up, its resolution being 
approximately determined by the satellite resolution. 

15 

This traditional approach uses the pixel hardware in a directly linked 
sense to the data. It ignores the fact that the centre of the actual field of 
view is not necessarily at the centre of the pixel. It ignores the fact that 
the field of view is not described by the pixel geometry, the actual field of 
20 view is more likely to be an ellipse and may vary in size. It ignores 
overlap between sensor values or the relationship between them. These 
shortcomings of data processing are not normally significant when dealing 
with very high resolution data, say in the order of metres, but become 
much more of a problem if the sensor resolution is over kilometres. 

25 

The present approach is to look at the entire imaging process and produce 
a more rigorous definition of remote sensing processing. Central to the 
idea is changing ground area boundaries from the prior art arbitrary pixel 
display to a properly geometrically defined set of geographic areas. The 
30 geometric boundaries may constitute real boundaries such as coastlines or 
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fields, or simply by arbitrary grids of rectangular areas of fixed or 
variable size. But in every case the boundaries must yield a fixed and 
consistent metric or coordinate system. The sensor then provides 
observed values over these boundaries either at the same time or at 
5 different times, with or without overlapping fields of view. We prefer 
overlapping fields of view - which is quite contrary to conventional 
thinking. 

If one assumes uniform values for each area within its boundary and over 
10 the time interval of the measurements being taken, then for each sensor 
measurement field of view these real responses are integrated according to 
the antenna response pattern. This gives sensor measurements as function 
of areal responses. Given all these related measurements it is then 
possible to solve for the whole geometric system to give the ground values 
15 for each defined area. 

In the prior art, the main point is that remote sensing samples are 
projected "raw" onto pixel hardware without there being a proper 
mapping between them. We can remap the antenna pattern into land and 

20 sea areas (or other parameters) and can map onto any co-ordinate system 
or geographical base. In other words the post sensor processing should 
remap the data onto a defined GIS or similar. Instead of computing, say, 
nine samples into two regions (as discussed in a later example) there is no 
reason why n samples should not be computed into n regions. A chip 

25 could be loaded with the GIS and all samples pushed through it as a space 
time data stream with each sample positioned within the GIS. 

It will be appreciated that in prior art conventional Remote Sensing (e.g. 
using satellite data) established practice is to dislike irregular fields of 
30 view (FOV) (i.e. all fields of view from satellites are irregular owing to 
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the satellite motion, scanning characteristics, the curvature of the earth 
and antenna gain functions), the solution being to crop down the data to 
give a central region of the FOV and say that that is the value for a pixel. 
Conventional thinking is to disregard, and even dislike, over-sampling 
5 (which is common, and becoming more so with modern sensing systems). 
In our view conventional thinking samples/fits RS data too soon to some 
standard co-ordinate framework - i.e. before it is weighted to compensate 
for contamination of signals by things from adjacent pixels (on the 
ground). The prior art fails to exploit the full set of available 
10 observations. 

We, on the other hand, believe that we achieve significantly better end- 
user oriented products from almost any digital data-base RS/GIS system, 
without awaiting "next-generation" sensors. 

15 

In accordance with a fourth aspect, the invention provides a method of 
improving the resolution of an imaging system which generates original 
pixel signals corresponding to pixels of a pixellated detection field of 
view or attributable to pixels of a virtual pixellated detector field of view, 
20 the method comprising: 

taking the original signals and operating on them with a weighting 
function so as to generate processed signals for each mapping region of a 
predetermined map, the weighting function attributing a contributing 
25 weight to each original pixel signal so as to determine a processed signal 
for each mapping region. 

Preferably the weighting function applies the antennas gain 
function/sensitivity function of the imaging system to the original pixel 
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signals to at least partially determine the processed signal for each 
mapping region. 

Preferably the method comprises processing the original signals to obtain 
5 a plurality of values for at least some pixels of the original pixellated field 
of view (and possibly for all). 

According to a fifth aspect, the invention provides an image processing 
apparatus comprising input signal receiving means; processed signal 

10 output means; and computational means; the computational means being 
adapted to take a set of original input signals from the input signal 
receiving means and of processing them by applying a weighting function 
which in use is used to generate an output signal for a predetermined 
mapping region, the mapping region being input to the computational 

15 means or stored there; the computational means being capable of applying 
respective appropriate weighting to the original input signals of a set of 
signals representative of an image to be processed to generate each 
respective output signal corresponding to each respective mapping region 
and of outputting signals corresponding to the mapping regions to the 

20 processed signal output means. 

According to a sixth aspect, the invention provides an image processing 
system comprising image capturing means and image processing means; 
the image capturing means capturing or sensing an image signal for each 
25 pixel of a pixellated image field of view, and the image processing means 
comprising image processing apparatus in accordance with the preceding 
aspect of the invention; the arrangement being such that in use the image 
capturing means obtaining an image of original signals for each of its 
pixels and the image processing means processing them to create output 
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image processed signals translated into the mapping regions of the image 
processing apparatus. 

Preferably the system is capable of oversampling an observed scene to 
5 create a plurality of sensed original image signals for elements with the 
scene (possibly by having spatially overlapping original images, a 
plurality of which contain the elements, or by having temporarily 
overlapping original scene images, or both spatially and temporarily 
overlapping original scene, images), and of using the "redundant" data to 
10 improve the resolution of the system beyond the resolution of the image 
capturing means alone. 

The image capturing means may comprise a microwave antenna or an 
infrared receiver. This may be mounted on a remote sensing platform 
15 such as a satellite, aeroplane or ship. 

In accordance with a seventh aspect, the invention provides a satellite 
system, or other remote sensing vehicle or installation having one or more 
sensors having an overlapping field of view so as to produce overlapping 
20 unprocessed images or data samples, and means for processing the images 
or data samples using a method in accordance with any one of the first, 
second, third or fourth aspects of the invention. 

The system may comprise a plurality of sensors which sense infrared, 
25 microwave, visible or other radiation, such as a microwave antenna. 

Looking at it one way, the invention lies in the use of overlapping image 
data from an imaging system to improve the resolution of the imaging 
system. By providing a satellite having a number of overlapping fields of 
30 view, and using the overlapping images in combination, better resolution 
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can be allowed than using each image separately to derive information. 
This is achieved, in particular, by using a weighing function in 
combination with an accurate map of the size and/or shape of at least 
some, and preferably substantially all, features that should be being 
5 imaged to improve resolution. 

The invention is especially applicable to the processing of images which 
exhibit areas with different spatial characteristics, and can result in more 
accurate knowledge of the value of the characteristics near a boundary. 
10 A map indicating where the boundary of the area is helps to compensate 
for erroneous effects due to boundary contamination. As one example, 
this may allow ships to be located in an area adjacent a coastline by 
producing accurate information in that area. Changes over time can be 
used to detect movement of a ship into or out of that area. 

15 

Part of a particular embodiment of the invention relies upon the condition, 
or assumption, that within two bounded areas or mapping regions points 
in one mapping region have a measured value/signal (representative of a 
characteristic such as temperature) that is constant within the mapping 
20 region, or at least "constant" in the sense of being closer to other 
characteristic values (signals for other parts within the mapping region) 
than to points in the other bounded area or mapping region. 

There will now be described, by way of example only, one embodiment of 
25 the present invention with reference to the accompanying drawings of 
which: 

Figure 1 illustrates the configuration of the Special Sensor 
Microwave Imager (SSM/I) showing the scanning motion of the 
30 sensor; 
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Figure 2 shows the arrangement of nine overlapping data footprints 
used to produce a set of linear equations; 

Figures 3, 4 and 5 show the three vertically polarised test images 
using the "raw" scan/back co-ordinate system; 

Figure 6 illustrates the land map of boundaries derived from UK 
admiralty charts; 

Figure 7 is a plot of geolocation error against relative position of 
boresight and boundary; 

Figure 8 is a plot of change of height against height itself on the 
same axis as Fig 7; 

Figures 9-11 illustrate a photographic view of the test scenes used; 
Figures 12-14 illustrate corresponding SSM/I images for figures 9- 

li; 

Figures 15-23 illustrate results of processing of test images; and 

Figure 24 is an overview of a method in accordance with one 
embodiment of the invention. 

A method in accordance with one embodiment is shown in schematic form 
in Figure 24. 
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In a first step, a set of data samples are acquired to form a sample view or 
two dimensional image. The data samples may comprise various 
overlapping snapshots of parts of an area of a surface. These may be 
acquired by an antenna having a known gain function and a sensitivity to 
5 the parameter of interest. 

In a second step, a map or transformation of each snapshot on the surface 
is constructed by processing the effective field of view of the sensor in 
combination with a knowledge of the shape of the surface. This enables 
10 the actual footprint for each snapshot to be calculated using data about the 
surface profile. 

A set of, say n, data samples forming an overlapping grid are then 
selected, and each data sample is allocated a weight V^.... V n . 

15 The calculation of the weights is achieved from a combination of 
knowledge of the expected parameters of features of the surface being 
measured. For example, the surface is divided into bounded areas or 
mapping regions. Each area is assumed to be uniform and homogeneous 
and is allocated a variable p (corresponding to its spectral value). The 

20 weighting w for each data sample depends on the location of the footprint 
relative to these bounded areas. 

The n data samples and the weightings V t V m are arranged in a matrix 

and, by suitable mathematics processing, the values of the unknown 
25 variables allocated to each area are determined. Returning to the map, 
these calculated variables can then be used to form an image point or 
pixel at the centre of the n data samples. 

The process can then be repeated using blocks of adjacent data samples to 
30 build up values for adjacent image points or pixels in a similar manner 
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(by calculating the allocated unknown variables from the new data) until a 
complete processed image of the whole surface has been constructed. 
Alternately, a set of footprints carry the whole surface of interest could be 
proceeded at once to substantially simultaneously produce pixel values. 

5 

Of course, extracting the variables from the datasets depends upon the set 
of equations in terms of measured data and weightings being soluble. The 
method may therefore include further processing steps to test for 
computability and reliability of the processing. 

10 

In one arrangement, the weightings are allocated to respective linear 
equations, one per data sample, according to the overlap between the 
footprint for the sample and the variables p allocated to each area. This 
also takes into account the sensor gain function, and can be readily 
15 achieved by conjugating the gain function with the surface area 
overlapping a footprint. 

One source of passive microwave data samples which can be used to 
measure surface temperature is the special Sensor Microwave Images 
20 (SSM/I) on the United States Defence Meteorological Satellite program 
range of spacecraft. The instrument measures incident microwave 
radiation emitted or reflected by the earths surface with typical 3dB 
footprints of between 69 x 43 km and 15 x 13 km, dependent upon 
frequency. 

25 

With such large footprints, coastal zones must (using prior data extraction 
methodology) be eliminated from the areas for which many physical 
parameter would be retrieved to avoid problems due to contamination of 
footprints for coastal zone data samples (footprints covering both land and 
30 sea) . 
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An important example of the problem of this coastal contamination occurs 
in the retrieval of surface wind speeds from the microwave data. 

5 The SSM/I instrument is constructed to continuously rotate, completing 
31.6 revolutions each minute. For approximately one third of each 
rotation (51.2 degrees to either side of the direction of spacecraft motion) 
the instrument measures the incoming radiation. The axis of the rotation 
is maintained to ensure a constant incidence angle between the direction of 
10 measurement and the surface normal vector at the point where the 
measurement direction intersects the Earth's surface. The combination of 
this conical scanning arrangement and the motion of the satellite results in 
data being collected on a succession of curved swaths across the Earth's 
surface each swath being 1400 km long (Figure 1). 

15 

The 85.5 GHz measurements are made every 4.22 milliseconds along the 
swath, resulting in 128 measurements spaced 12.5 km apart. The 
spacecraft itself moves 12.5 km forward during the 1.9 seconds required 
for one full rotation of the instrument. The high resolution 85.5 GHz 
20 data is sampled for every swath giving a 12.5 by 12.5 km resolution. The 
lower frequency channels are sampled every 8.44 milliseconds on 
alternate scans (designated A-scans), giving a sampled resolution of 25 km 
by 25 km. 

25 The brightness temperature measured by the satellite in a given channel is 
affected by a number of factors: the instrument calibration; extraneous 
signals due to cross polarisation effects and radiation emanating from 
parts of the spacecraft and from outer space; and the signal from the 
target area entering the instrument at angles close to the direction of view 

30 (along the boresight). Calibration and extraneous signal effects may be 
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removed using a standard pre-processing technique which results in a data 
sample comprising a "brightness temperature" measurement at the 
satellite which is supposed to represent as closely as possible the net 
radiation input for a narrow range of angles about the boresight. This 
5 brightness temperature should be equal the convolution of the antenna 
gain characteristics with the pattern of incoming radiation. A gain 
function which represents the responsiveness of the instrument to 
radiation entering at different angles to the boresight was experimentally 
determined pre-flight. 



A particular satellite brightness temperature measurement may be related 
to a pattern of microwave emission from the surface (and atmosphere) of 
the Earth by: 



15 T b = jh(t)^G(e(t)+Q,Mt) +^)T b (X(Q4),Y(Q^),t)cos<SfdQd^dt 

where t is the time relative to the measurement time, h(t) is the impulse 
response of the instrument, G(®(t) is the normalised gain function for the 

20 instrument in the direction (0,<j>) relative to the direction of the boresight 
at the measurement time, <d(t),Q(t) describe the movement of the boresight 
during the measurement period and (X(Q,§),Y(Q,§))is the projection of the 
viewing direction (9,<j>)onto the surface of the Earth. Assuming that 
brightness temperatures for any sensed area vary slowly compared to the 

25 time scale of the measurement and reversing the order of integration: 



10 
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+» 

where G(6,$) = jh(t)G(e(t)+Q,&(t)+$)dt 

where C?(6 ,<j>) is the antenna response integrated over the measurement 
interval. Given the consistent motion of the SSM/I instrument this 
5 function is assumed to be independent of the measurement time. 

The footprint associated with the true antenna response G(0,<j>)is known as 
the Instantaneous Field of View (IFOV). The footprint for the time 
integrated G<0,<j>)is called the Effective Field of View <EFOV). The 
10 projection of the EFOV onto the surface of the Earth results in an 
elliptical footprint, with the shorter axis parallel to the scan. Since the 
sampling distance at each frequency is shorter than the dimensions of 
these projected footprints they overlap. This overlap blurs the microwave 
image and extends the coastal contamination problem. 

15 

In the example described hereinafter, to obtain accurate date even at 
coastal regions the antenna gain function is convolved with a simple 
microwave emission model. The model uses GIS data to determine the 
land sea boundary and then assumes that all land areas within a small 

20 localised region have the same brightness temperature and that the sea 
within that region has an independent, but similarly uniform, brightness 
temperature. Convolving the antenna gain function with the model over 
the footprint associated with a given pixel results in a linear equation 
relating the brightness temperature measured at the satellite to the land 

25 and sea brightness temperatures assumed by the model. Repeating the 
convolution process for each of nine pixels forming a 3x3 square about a 
particular point as shown in Figure 2 results in an over-specified system 
of linear equations relating two unknowns (the modelled land and sea 
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brightness temperatures which are assumed to have the same respective 
values within each footprint) to nine known values (the measured 
brightness temperature for each pixel). These equations are solved using 
a least squares optimisation technique to obtain estimates for the actual 
5 land and sea brightness temperatures present within a combined footprint 
of the nine pixels. 

As described hereinbefore, the time-integrated antenna pattern is 
convolved with a simple model which employs only two brightness 

10 temperatures to describe the combined microwave emission from surface 
and atmosphere over land and sea areas respectively within the footprint 
of a particular measurement (i.e. only two mapping regions, each assumed 
to have a constant brightness temperature). The model is derived using 
GIS data to determine the coastal position and may be mathematically 

15 expressed as: 

T b (x,y) = T, and L(x,y> + T sea (l-L(x.y)) 

Where L(x,y) = 1 if (x,y) is a point on land 

20 = 0 if (x,y) is a point in the ocean 

T ]and = the brightness temperature of the land 
T sea = tne brightness temperature of the ocean 

25 Here land and sea brightness temperatures are assumed to include the 
effects of the atmosphere intervening between a given point on the surface 
and the sensor. 

The brightness temperatures seen by the satellite may be obtained by 
30 convolving the antenna gain function over the model substitution gives: 
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^=J|G<e,<p){L(m4>>,m<i>))^^ 

=ccT lmd + (l-a)T sea 
where a = JJ(? (9, §)L(X (9, (J>), 7(9 cos<j>rf&*}> 

5 

The above equation may be applied for each of the nine brightness 
temperature measurements corresponding to the 9 pixels surrounding the 
given point. Assuming uniform land and sea brightness temperatures 
within the 9 overlapping footprints, this gives rise to a set of simultaneous 
10 linear equations: 

T bij = ai,jT land + (l-ccij) T sea i = 1,2,3 j = 1,2,3 

With nine equations and only two unknowns (T land , T sea ), this system is 
15 clearly overspecified and will in general not have an exact solution. Some 
form of optimisation technique must be used. Restructuring: 

Tbi.j = ccy (T land - T sea ) + T sea 

20 shows that a linear regression of the brightness temperatures T b against 
the weights a will yield values for T land -T sea and T sea , which corresponds 
to the slope and intercept of the fitted line respectively. 

This retrieval results in average land and sea brightness temperatures for a 
25 region consisting of the combined footprints from nine pixels 
approximately 87 by 78 km square (defined by the 3dB level for the 37 
GHz footprint) as opposed to 37 by 28 km for the original pixel 
measurements. (Although it should be noted that T b values are estimated 
for each pixel position on the original 37 GHz data grid (i.e. every 25 
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km)). Thus resolution is sacrificed to gain the information needed to 
separate land and sea brightness temperatures. 

The accuracy of the retrievals will be affected by the veracity of the 
5 assumption that the surface emission for an area covered by nine 
measurement footprints may be represented by only two uniform values 
for brightness temperature. Since this assumption is fundamental to this 
embodiment of the technique any resulting errors will represent an 
absolute bound on the accuracy of retrieval. These errors were partially 

10 constrained in this work by the use of cloud-free imagery of satellite 
frequencies which thus limited atmospheric, although not surface, 
variations. Other factors affecting the accuracy of the retrievals relate to 
the precision with which the relationship between surface emission 
patterns and temperature measurements at the satellite is modelled. 

15 Errors associated with these factors must be minimised if realistic land 
and sea brightness temperatures are to be retrieved. 

The method has been applied to three 37.0 GHz vertically polarised 
SSM/I images from the DMSP F13 satellite for the region around the 
20 southern end of the Red Sea and the Gulf of Aden (5-25N, 35-55E). This 
region was chosen because desert coastlines should present a less complex 
test case than would vegetated ones. 

The images were selected to meet the following criteria: 

25 

The image had to be as cloud free as possible. This was necessary 
both to provide a simple and ambiguous test for the retrieval 
technique itself and to facilitate any geolocation correction. 
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Coincident visible data from the Operational Linescan System 
(OLS; an instrument located on the same satellite) should be 
available. These data would be used to facilitate an investigation 
into possible OLS-SSM/I co-location and to provide additional 
5 higher resolution information about the studied scene. 

The OLS images were examined on the SPIDER (Space Physics 
Interactive Data Resource) World Wide Web site and once selected were 
ordered from the US National Geophysical Data Centre (NGDC), which 
10 made the data available by FTP. The corresponding SSM/I image data 
(see Table 2) were extracted from 8 mm tapes from the CRS archive and 
decoded using standard packages. This software generated "brightness 
temperature" images, computed from the actual satellite temperature 
measurements using a standard pre-processing technique. 

15 

Table 2: Test Images 



Date 


Julian Day 


Orbit 


Times 


23 March 1996 


83 


515354 


12:54-14:38 


24 March 1996 


84 


516768 


12:42-14:26 


31 March 1996 


91 


526667 


12:59-14:42 



20 

Figures 3, 4 and 5 show the three 37 GHz vertically polarised SSM/I test 
images using the "raw" scan/track co-ordinate system. The coastal 
regions are delineated by white lines. 
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The coastline for the study area was digitised from UK Admiralty charts 
to an accuracy of the order of 1 km, this data being used to create a land 
sea mask of boundaries on a lat./long grid at a resolution of 0.02° (Figure 
6). 

5 

In the method of the present example of the invention, the precision with 
which the satellite field of view is projected onto the surface of the Earth 
is important, since the convolution operation is performed at a spatial 
scale of less than 1/8 of a pixel. Unfortunately, the satellite ephemeris 

10 information in the SSM/I data is not supplied in a form readily applicable 
to this problem. The data do, however, include geolocation information 
for each pixel which may be reversed through the satellite-Earth 
projection process to obtain some of the necessary parameters. The 
positional data do not display the required sub-pixel level of accuracy and 

15 must be improved through a manual or automatic re-registration of the 
microwave image before being employed. 

The sensitivity of the retrieval technique to geolocation errors may be 
investigated by convolving the antenna gain function with models 

20 consisting of a straight boundary at different positions. As before, the 
convolution results in a weight, a, which indicates the relative 
contribution made on one side of the boundary. If one is trying to 
retrieve values for the other side of the boundary, this weight may be 
regarded as the error due to contamination. For example, if cc is deemed 

25 to refer to the contribution of the land brightness temperature to the 
mixed pixel value, that weight may also be regarded as indicating the 
relative error which the presence of land introduces into the pixel 
brightness temperature if that value is viewed as an ocean brightness 
temperature. 



30 
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The difference between the weights derived for two different, but closely 
spaced, boundary positions give an indication of the error introduced into 
the modelling process by a faulty geolocation. Figure 7 plots both the 
spatial derivative (the geolocation error) against the relative position of 
5 boresight and boundary. Figure 8 plots the relative geolocation error 
(change in weight divided by the weight itself) against the same axis. 

In order for the retrieval technique to actually improve anything, the error 
in modelling the antenna response to the a GIS model must be less than 
10 the "contamination" error the technique is trying to remove. An 
examination of Figures 7 and 8 leads to two important conclusions: 

Geolocation errors of greater than about 2 km will lead to errors 
which comprise a significant portion of the effect to be removed. 

15 

For certain positions of the coast with respect to the boresight the 
technique will be ineffective for any reasonable geolocation error 
since the effects of a displaced location far outweigh those of the 
combination error itself (see Figure 8) . 

20 

Sea surface wind speeds and other important geophysical parameters can 
be derived from polarisation differences. The effect of geolocation errors 
on these values may be determined thus: 

25 

T v -T h =a v T rJmd + (l-a v )T v ^ -<x h T hJand - (\-a h )T hjsea 

= a v iT viand -T hiand ) + (l-cL v )(T vjea -T h jJ + (a v -a h ){T hJand -T krS J 
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Here the v and h refer to vertical and horizontal linear polarisations 
respectively. This equation reveals that polarisation differences vary 
across a coastline in a manner very similar to brightness temperatures. 
The difference lies in the third term on the right hand side of the equation 
5 which represents the effects of any asymmetry between the vertical and 
horizontal instrument gain functions. Unfortunately there is known to be 
a discrepancy of several kilometres between the maxima of the vertical 
and horizontal gain functions (as projected onto the ground). Thus the 
third term, while still small, has some significance. 

10 

There is considerable error in the locational accuracy of both OLS and 
SSM/I imagery as can be seen in Figures 9-14. Figures 9, 10 and 11 
show the visible (OLS) view of each scene while figures 12, 13 and 14 
show the corresponding 37 GHz vertically polarised SSM/I images. In 
15 these diagrams, the data are shown on a (0.020 resolution) 
Latitude/Longitude grid, with a digitised coastline overlaying the pixel 
data. The OLS data seem to be shifted to the West, whereas the SSM/I 
images are displaced to the North west. 

20 The strategy was carried out using the SSM/I imagery for three 85 GHz 
images corresponding to the three test windows on different days (day 1, 
day 2, day 8). The resulting transformation coefficients were then applied 
to the original imagery and the results for first, second, and third order 
polynomials for the day 1, day 2 and day 8 respectively are shown in 

25 Figures 15 through 23. The illustrations are overlaid with the digitised 
coast outline to illustrate the accuracy of the fit. 

The first order transform coefficients are given in Table 3. 
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Table 3: First order transform coefficients for the three SSM/I 
images. 



Date 




x' 


y' 


Julian 
Day 


Day 1 


const 


8.1049 


9.8219 


83 




X 


0.9883 


0.0107 


83 




y 


0.0056 


0.9892 


83 


Day 2 


const 


2.5049 


8.4483 


84 




X 


1.0090 


0.0018 


84 




y 


-0.0027 


0.9997 


84 


Day 8 


const 


10.0395 


10.3634 


91 




X 


1.0015 


-0.0018 


91 




y 


-0.0049 


0.9991 


91 



5 From the constant term we can see that the geolocational error caused by 
shift alone is of the order of 8 to 10 pixels (approx. 16-20 km) in both the 
x and y directions. With day 84 being the exception with a shift of only 
2.5 pixels in the x direction. The errors of the fit of the GCPs (i.e. 
coastline) to the polynomial are given in Table 4. For the 83 and 91 

10 images the improvement in fig going from first to second order is 
considerable, whereas from second to third is negligible. For day 84 the 
improvement from first to second order is less effective, whereas the 
progression from second to third order is much more effective. 
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Table 4 Root Mean Square Errors for the Ground Control Point fit to 
the Transformation Polynomial. 

RMS Error 





li UIll 

GCPs 




X 


y 


i otai 


23 March 


219 


i 


2.559 


3.035 


3.970 






2 


1.127 


1.374 


1.777 






3 


1.068 


1.051 


1.498 




214 


1 


2.564 


2.087 


3.305 






2 


2.323 


1.374 


2.699 






3 


1.215 


1.017 


1.585 




347 


1 


2.069 


1.325 


2.458 






2 


1.460 


1.132 


1.850 






3 


1.419 


0.996 


1.733 



5 

Following the geolocation techniques described above a further means of 
improving image geolocation may be considered. In this method pixel 
locations are varied so as to optimise the goodness of fig criterion in the 
linear regression used to derive the sea and land temperatures. The 
10 retrieval process may be completed at each location for a set of possible 
geolocation displacements. Defining: 

a(bX,bY) = j]"G(e,<}>)L(X(e,<j>) +bX,Y(Q4) +6Y)cos<^dBd^ 

15 for a possible geolocation error (SX, 6Y) and substituting into the above 
equation yields: 

20 Restricting the possible displacements to points on a grid (i.e. 5X and 8Y 
are integer multiples of some small displacement) results in a set of 
overspecified linear systems. For example, if a 5 by 5 grid of 
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displacements is defined, 25 systems of nine equations each will result. 
The displacement which causes the retrieval to generate the smallest root 
mean square error in the linear regression may now be assumed to 
correspond to the true location of the nine pixel set used for the particular 
5 retrieval. Being associated with the correct location, the selected 
regression coefficients should yield the best possible values for the land 
and sea brightness temperatures. 

PRACTICAL IMPLEMENTATION OF THE METHOD 

10 

The first tasks in the implementation of the retrieval technique is to 
generate a set of weights a based on the convolution of the gain function 
with the land-sea scene. The gain function must be correctly projected 
onto the Earth's surface and so depends on the satellite-Earth geometry 
15 associated with each pixel. To accomplish this, detailed information 
concerning the movement of both the satellite and the direction of the 
SSM/I sensor is required. There are a number of formats in which SSM/I 
data may be supplied. The F13 SSM/I data used for this study was 
originally provided by the US Fleet Numerical Meteorology and 
20 Oceanography Center (FNMOC). These data contain both ephemeris 
information for the satellite and a latitude/longitude location for each 
pixel. While the decoding software provided with the FNMOC data tapes 

does not extract satellite ephemeris information, communications with 

NGDC establish the necessary changes to the code. Unfortunately, the 
25 ephemeris data were not usable; the times for the satellite position 

measurements did not coincide with the times for the corresponding 

temperature data. 

The only information within the FNMOC data set from which the 
30 movement of the satellite with time could be readily derived was the 
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location data for the pixel grid. Provisionally ignoring the movement of 
the satellite during the course of a single scan, all pixels within that scan 
may be assumed to be equidistant from the satellite location. This 
assumption allows that location to be simply derived using spherical 
5 trigonometry. Given the satellite location, the measurement direction for 
each pixel in the scan may be calculated and the antenna footprint 
projected onto the ground. Other parameters for the projection such as 
satellite altitude and incidence angle between the observation vector and 
the surface of the Earth must then be assumed to be close to their 
10 published mean values. This assumption is more reasonable for the 
incidence angle, which does not vary to a significant degree than it is for 
altitude; the altitude of the F13 satellite varies from 840 km to 875 km; 
the variation of incidence angle for F13 is not available, but for previous 
spacecraft this parameter has varied by less than 1.5°. 

15 

Given the need to make such approximations, the projection of the 
boresight direction for a measurement onto the ground does not exactly 
match the pixel location given in the data. The latter position (as 
corrected by any geolocation enhancements) is assumed to be more 
20 reliable, and the location of the centre of the projected footprint shifted 
accordingly. 

With the projection parameters in place, the footprint must be projected 
onto the ground. Although projection software had been obtained from 
25 the University of Bremen, this was restricted in its application and 
contained a number of significant approximations. The projection 
function was thus rederived from basic principles using plane and 
spherical trigonometry and implemented as a subroutine for use by the 
retrieval programs. 



30 



WO 00/42385 



PCT/GBOO/00063 



35 

Once the correct footprint projection function has been established, 
calculating the weights for each pixel is relatively straightforward. The 
time- integrated (Effective Field of View) antenna pattern is read in as an 
array of values on a 0.1° grid. The convolution is approximated by a 
5 summation, with each point in the pattern array being included in the 
summation if and only if that particular viewing direction intersects with a 
portion of the Earth's surface deemed to be land by the GIS mask. The 
net result of this process is an array of weights, and, each weight value 
corresponding to a single SSM/I 37 GHz pixel. 

10 

If the optimal-fit geolocation improvement technique (described in Section 
6.3) is to be employed, the above process must be repeated once for each 
possible displacement between the given pixel location and its actual 
position. These displacements were taken from a 5 by 5 grid, consisting 

15 of possible displacements of -0.04°, -0.02°, 0, 0.02° or 0.04° in either 
ordinate (latitude or longitude), resulting in twenty-five possible values 
for a for each pixel. Correction using a much larger 21 by 21 array with 
the same angular resolution of 0.02° between displacements was attempted 
in an effort to eliminate the necessity for the image-matching geolocation 

20 enhancement step. 

The computation of weights is a relatively slow process. For a simple 
retrieval a 64 by 64 image took 50 minutes on one of the CRS Silicon 
Graphics Indy 2 workstations. The same image using the 5 by 5 array for 
25 each pixel needed for geolocation enhancement took 12 hours to compute, 
and, even with some additional optimisation to the algorithm, the 21 by 
21 array took 48 hours. In future it should be possible to restructure the 
retrieval algorithm to achieve running times of one quarter to one third of 
this total, for example by extending the GIS mask to differentiate between 
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potentially coastal pixels and those which must be either 100% land or 
100% ocean. 

Once the weights have been computed, they may be combined with 
microwave brightness temperature data for the 9 pixel square surrounding 
each pixel in the area of interest to form the dataset for a linear 
regression. If more than one weight has been computed for each pixel, a 
regression is performed for each possible displacement of the nine pixels 
(which are assumed to remain fixed with respect to each other). The 
coefficients returned by the linear regression with the smallest root mean 
square, error are used to calculate the land and sea brightness 
temperatures for the central pixel. The process is carried out for each 
pixel in the original grid spacing (25 km x 25 km). 

As is apparent to the skilled man, the method of the present invention is 
applicable to data extraction in any application in which a sensor with a 
known (or approximated or estimated) gain function is provided to 
produce an output corresponding to a sensed area. Suitable sensors which 
operate in such a manner include sonar hydrophones, microwave antennas, 
RF antennas etc. Each produce a convolved output data sample. 

Envisaged applications include trafficability, sea ice monitoring, 
scatterometry, moving vs stationary image recognition, agricultural land 
use (each field perhaps defining a boundary), irrigation management, 
ocean front variations, and coastal zone monitoring. 

It is also envisaged that more abstract applications will fall within the 
scope of the invention, the surface perhaps being an image which is 
"observed" by a sensor, such as an image taken by a camera (e.g. digital 
or video camera) or optical sensing element. Possible sensors for use in 
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the method may cover radar to sonar, from visible to infrared or vertical 
sounding etc. The important feature is knowledge (or approximation) of 
the sensor gain function and use of this function to extract more accurate 
data using a knowledge of "surface" boundaries. 

5 

In summary, the present invention provides, in one embodiment, a method 
of improving the extraction of data land/sea images observed by satellite 
base sensors. 

10 Given sensor values, VI to Vm, and areas Al to An, where each V 

is an integral of Al....An, and assuming each area A is precisely 
geometrically defined, and internally homogeneous; assuming that the 
sensor spatial integration V is also precisely geometrically defined; and 
assuming that the entire geometric frame of reference is consistent, then 

15 providing m is greater than n we have a soluble system of linear 
equations. It may be that the problem naturally provides for the last 
constraint, but it can be ensured that m is greater than or equal to n in 
three ways, first by degrading in the spatial dimension; or second by 
degrading in the temporal dimension; or third, both together. 

20 

In the case of a coastal zone study it has been assumed that each dataset is 
a single time slice, and nine sensor values were taken, VI -9 to arrive at 
two unknowns, al and a2, in this case land and sea values. If the time 
interval were less than the variability of the parameter being measured, 
25 then it is possible that the V equations could be derived from successive 
time slices, rather than from the same time slice. 

Regarding the source of "additional information", we would say first that 
the addition of the geometric definitions of A and their specified 
30 relationships with V provide the required data for more accurate 
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interpretation of V. In one sense the information contained in V is 
obviously not increased, but the procedure provides a means of extracting 
the maximum amount of information from the original data. 

5 It will be appreciated that in addition to being applicable to image 
processing, and remote sensing (not necessarily of an image), the 
invention is more generally applicable to signal processing and protection 
for such is envisaged. 
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CLAIMS 

1. A method of extracting information from remotely sensed data 
comprising the steps of acquiring a plurality of data samples from at least 

5 one sensor, each data sample corresponding to a respective footprint and 
said one or more sensors being adapted to sense a parameter of a surface 
and having a known gain function, processing (reprojecting) the data 
samples to produce a respective footprint corresponding to the area of the 
surface contributing to each data sample, selecting a set of n bounded 

10 areas for the surface being sensed and allocating a variable to the area 
defined by each boundary, defining a weighting a for each data sample 
dependent upon the sensor gain function and the location of the footprint 
for the respective data sample relative to the bounded areas of the surface, 
constructing a set of at least n equations from said weighted data samples, 

15 and solving said equations to determine the values for the variables 
allocated to respective bounded areas. 

2. A method according to claim 1 in which each data sample 
comprises a convolution of the sensor gain function with the signal from 

20 the footprint on the surface which is incident upon the sensor. 

3. A method according to claim 1 or claim 2 in which the equations 
comprise linear equations. 

25 4. A method according to any preceding claim in which the variables 
correspond to points in an image of a surface constructed from an array of 
data samples from different positions on the surface. 
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5. A method according to any preceding claim in which each variable 
allocated to a bounded area corresponds to the parameter to be sensed 
(e.g. surface temperature). 

5 6. A method according to any preceding claim in which the variables 
comprise surface temperature measurements and the sensor comprises a 
microwave antenna adapted to detect radiation emitted from or reflected 
off the surface. 

10 7. A method according to any preceding claim in which more than one 
set of linear equations is used and each linear equation in a set may be 
allocated a weighting, with different weightings used for each set. 

8. A method according to any preceding claim in which the sensor is 
15 adapted to receive information radiated or reflected from a footprint on 
the surface to produce a data sample and the linear equations may then be 
calculated from data samples corresponding to a number of such footprints 
for different surface areas. 

20 9. A method according to claim 8 in which the footprints for two or 
more of the data samples used to construct the set of linear equations 
overlap at least partially. 

10. A method according to claim 9 in which the bounded areas define 
25 the boundaries of areas of land and ocean at a coastline. 

11. A method according to any preceding claim in which each bounded 
area is smaller than the size of each footprint for the data samples. 
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12. A method according to any preceding claim in which each data 
sample used to construct the set of linear equations corresponds to a 
footprint covering a different but overlapping area of the surface. 

5 13. A method according to any preceding claim in which the 
boundaries are defined by reprojecting the data samples onto a GIS map of 
the surface. 

14. A method according to any preceding claim in which the weights, 
10 a, are based on the convolution for the gain function of the receiver with 

the boundaries of the area surface sensed. 

15. A method according to claim 14 in which the convolution is 
approximated by a summation. 

15 

16. A software program for implementing on a computer adapted to 
perform a method in accordance with any one of claims 1 to 15. 

17. A method of processing an original set of signals to create a 
20 processed image comprising: 

taking a set of original signals derived from one or more image sensors, 
the signals being representative of an image to be image processed; 

25 having a spatial map upon which the original image signals are to be 
translated, the map having a plurality of mapping regions; 

having weighting coefficients for each original signal in the set of original 
signals corresponding to the weight to be given to each individual original 
30 signal associated with any particular mapping region, and 
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creating processed image signals for each mapping region, the processed 
image signals for individual mapping regions being determined by 
applying the weighting coefficients to the set of original signals for each 
5 individual mapping region so as to create a combined, weighted, 
processed signal associated with each mapping region. 

18. A method of processing according to claim 17 in which there are at 
least as many original signals in a set as there are spatial mapping 

10 regions. 

19. A method according to claim 17 or claim 18 in which the weighting 
coefficients are associated with the directional sensitivity of the sensor(s) 
used to capture the set of original signals. 

15 

20. A method according to any one of claims 17 to 19 in which the 
processed signal for each mapping region is determined by assuming it to 
be a constant value and solving a redundant, or solvable, number of 
simultaneous equations which each have the processed signal as one 

20 parameter, and weighted values derived from the original signals as other 
parameters, the weighting and the original signals being known. 

21. A method according to any one of claims 17 to 20 in which the 
mapping regions are representative of real physical features known to be 

25 present in the scene being viewed by the image-gathering apparatus. 

22. A method of improving the resolution of an imaging system which 
generates original pixel signals corresponding to pixels of a pixellated 
detection field of view or attributable to pixels of a virtual pixellated 

30 detector field of view, the method comprising: 
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taking the original signals and operating on them with a weighting 
function so as to generate processed signals for each mapping region of a 
predetermined map, the weighting function attributing a contributing 
5 weight to each original pixel signal so as to determine a processed signal 
for each mapping region. 

23. A method according to claim 22 in which the weighting function 
applies the antennas gain function/sensitivity function of the imaging 

10 system to the original pixel signals to at least partially determine the 
processed signal for each mapping region. 

24. A method according to claim 22 or 23 in which the method 
comprises processing the original signals to obtain a plurality of values 

15 for at least some pixels of the original pixellated field of view. 

25. An image processing apparatus comprising input signal receiving 
means; processed signal output means; and computational means; the 
computational means being adapted to take a set of original input signals 

20 from the input signal receiving means and of processing them by applying 
a weighting function which in use is used to generate an output signal for 
a predetermined mapping region, the mapping region being input to the 
computational means or stored there; the computational means being 
capable of applying respective appropriate weighting to the original input 

25 signals of a set of signals representative of an image to be processed to 
generate each respective output signal corresponding to each respective 
mapping region and of outputting signals corresponding to the mapping 
regions to the processed signal output means. 
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26. A software carrier carrying software which when operational on a 
computer or network either provides an image processing apparatus 
according to claim 25 or operates the computer or network according to 
the method of any one of claims 1 to 24. 

5 

27. An image processing system comprising image capturing means and 
image processing means; the image capturing means capturing or sensing 
an image signal for each pixel of a pixellated image field of view, and the 
image processing means comprising image processing apparatus in 

10 accordance with the proceeding aspect of the invention; the arrangement 
being such that in use the image capturing means obtaining an image of 
original signals for each of its pixels and the image processing means 
processing them to create output image processed signals translated into 
the mapping regions of the image processing apparatus. 

15 

28. Apparatus according to claim 27 which is capable of oversampling 
an observed scene to create a plurality of sensed original image signals for 
elements with the scene and of using the "redundant" data to improve the 
resolution of the system beyond the resolution of the image capturing 

20 means alone. 

29. Apparatus according to claim 27 or claim 28 in which the image 
capturing means comprises a microwave antenna or an infrared receiver. 

25 30. Apparatus according to claim 27, 28 or 29 in which the image 
capturing means is mounted on a remote sensing vehicle such as a 
satellite, aeroplane or ship. 

31. A satellite system, or other remote sensing platform or installation 
30 having one or more sensors having an overlapping field of view so as to 
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produce overlapping unprocessed images or data samples, and means for 
processing the images or data samples using a method in accordance with 
any one of claims 1 to 20. 

32. A satellite system according to claim 30 in which a plurality of 
sensors are provided which sense infrared, microwave, visible or other 
radiation. 
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• 1 .* The scanning geometry of the SSMI instrument (from 
Hollinger et al. 1987) 
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Fig. 2: 



Example of the nine footprints used by the 
retrieval technique 
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Day 84 
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Brightness Temperature. K 

Fig. 3: SSM/I 37 GHz 
Vertically Polarised Image for Day 1. 
The coastal zone is indicated. 



Brightness Temperature. K 

Fig* 4: SSM/I 37 GHz 
Vertically Polarised Image for Day 2. 
The coastal zone is indicated. 



Day 91 
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Brightness Temperature. K 

Fig. 5: SSM/I 37 GHz 

Verically Polarised Image for Day 3. 
The coastal zone is indicated. 
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Fig. 6* The Coastal Mark digitized from Admirality Charts 
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r . 7 I Geolocation error relative to the location of the coastal 
boundary with respect to the boresight 
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Fig. 8 

Geolocation error as a fraction of coastal contamination error 
plotted against the relative location of the boundary with 
respect to the boresight 
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Coastline Overlay for Day 1 
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Fig. 13: 

SSM/1 85 GHz Vertically Polarised Image and Coastline Overlay 
for Day 2. 
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Fig. 14: 

SSM/I 85 GHz Vertically Polarised Image and Coastline Overlay 
for Day 8. 
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SSM/I 85 GHz Image for Day 1 Georectified with a 1st 
Order Polynomial. 
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SSM/I 85 GHz Image for Day 1 Georectified with a 2nd 
Order Polynomial. 
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Fig. 1 7: 

SSM/1 85 GHz Image for Day 1 Georectified with a 3rd 
Order Polynomial. 
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SSM/I 85 GHz Image for Day 2 Georectified with a 1st 
Order Polynomial. 
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SSM/T 85 GHz Image for Day 2 Georectified with a 2nd 
Order Polynomial. 
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Fig. 2 0: 

SSM/1 85 GHz Image for Day 3 Georectified with a 3rd 
Order Polynomial. 
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Fig. 21: 

SS/1 85 GHz Image for Day 8 Georectified with a 1 st Order 
Polynomial. 
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Fig. 22: 

SSM/I 85 GHz Image for Day 8 Georectified with a 2 
Order Polynomial. 
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SSM/1 85 GHz Image for Day 8 Georectified with a 3rd 
Order Polynomial. 
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